MD-Granting Medical Schools in the US, 2022-2023.

This Appendix presents data derived from the 2022-2023 Liaison Committee on Medical Education Annual Medical School Questionnaire-Part II.

Medical School Rankings-Bad for the Health of the Profession and the Public.

This Viewpoint discusses the limitations of medical school ranking in attracting a diverse student population and urges administrators to holistically communicate their mission, goals, and learning environment as an alternative strategy.

Assessment and Evaluation in Social Determinants of Health Education: a National Survey of US Medical Schools and Physician Assistant Programs.

BACKGROUND: Social determinants of health (SDOH) curricular content in medical schools and physician assistant programs are increasing. However, there is little understanding of current practice in SDOH learner assessment and program evaluation, or what the best practices are. OBJECTIVE: Our study aim was to describe the current landscape of assessment and evaluation at US medical schools and physician assistant programs as a first step in developing best practices in SDOH education. DESIGN: We conducted a national survey of SDOH educators from July to December 2020. The 55-item online survey covered learner assessment methods, program evaluation, faculty training, and barriers to effective assessment and evaluation. Results were analyzed using descriptive statistics. PARTICIPANTS: One hundred six SDOH educators representing 26% of medical schools and 23% of PA programs in the USA completed the survey. KEY RESULTS: Most programs reported using a variety of SDOH learner assessment methods. Faculty and self were the most common assessors of learners' SDOH knowledge, attitudes, and skills. Common barriers to effective learner assessment were lack of agreement on "SDOH competency" and lack of faculty training in assessment. Programs reported using evaluation results to refine curricular content, identify the need for new content, and improve assessment strategies. CONCLUSIONS: We identified a heterogeneity of SDOH assessment and evaluation practices among programs, as well as gaps and barriers in their educational practices. Specific guidance from accrediting bodies and professional organizations and agreement on SDOH competency as well as providing faculty with time, resources, and training will improve assessment and evaluation practice and ensure SDOH education is effective for students, patients, and communities.

Adapting the Admissions Interview During COVID-19: A Comparison of In-Person and Video-Based Interview Validity Evidence.

COVID-19 physical distancing limited many medical schools' abilities to conduct in-person interviews for the 2020 admissions cycle. The University of Toronto (U of T) Temerty Faculty of Medicine was already in the midst of its interview process, with two-thirds of applicants having completed the in-person modified personal interview (MPI). As the university and surrounding region were shut down, the shift was made in the middle of the application cycle to a semisynchronous video-based MPI interview (vMPI) approach. U of T undertook the development, deployment, and evaluation of the 2 approaches mid-admissions cycle. Existing resources and tools were used to create a tailored interview process with the assistance of applicants. The vMPI was similar in content and process to the MPI: a 4-station interview with each station mapped to attributes relevant to medical school success. Instead of live interviews, applicants recorded 5-minute responses to questions for each station using their own hardware. These responses were later assessed by raters asynchronously. Out of 627 applicants, 232 applicants completed the vMPI. Validity evidence was generated for the vMPI and compared with the MPI on the internal structure, relationship to other variables, and consequential validity, including applicant and interviewer acceptability. Overall, the vMPI demonstrated similar reliability and factor structure to the MPI. As with the MPI, applicant performance was predicted by nonacademic screening tools but not academic measures. Applicants' acceptance of the vMPI was positive. Most interviewers found the vMPI to be acceptable and reported confidence in their ratings. Continuing physical distancing concerns will require multiple options for admissions committees to select medical students. The vMPI is an example of a customized approach that schools can implement and may have advantages for selection beyond the COVID-19 pandemic. Future evaluation will examine additional validity evidence for the tool.

Undergraduate medical education in Nigeria: current standard and the need for advancement.

The post-independence era in Nigeria ushered in an array of fundamental structuring and development in all sectors of the Nigerian economy including medical education and training. This era saw the establishment of medical schools across the country which mirrored the medical curriculum of British universities. This paper dives into the general structure of undergraduate medical education in Nigeria, its historical background and how it compares with neighboring and distant countries. Since the undergraduate medical education curriculum has not seen significant modifications since conception, this paper presents the challenges of the existent structure to include biased admission process, emphasis on irrelevant pre-medical courses, paucity of of technologically-advanced teaching and learning aids, increased workloads of lecturers amongst others. Importantly, solutions and recommendations are prescribed in this paper, which if considered, may improve undergraduate medical training in Nigeria, and ultimately improve the standard of healthcare service provision in the country.

Downstream funding success of early career researchers for resubmitted versus new applications: A matched cohort.

BACKGROUND: Early career researchers face a hypercompetitive funding environment. To help identify effective intervention strategies for early career researchers, we examined whether first-time NIH R01 applicants who resubmitted their original, unfunded R01 application were more successful at obtaining any R01 funding within 3 and 5 years than original, unfunded applicants who submitted new NIH applications, and we examined whether underrepresented minority (URM) applicants differentially benefited from resubmission. Our observational study is consistent with an NIH working group's recommendations to develop interventions to encourage resubmission. METHODS AND FINDINGS: First-time applicants with US medical school academic faculty appointments who submitted an unfunded R01 application between 2000-2014 yielded 4,789 discussed and 7,019 not discussed applications. We then created comparable groups of first-time R01 applicants (resubmitted original R01 application or submitted new NIH applications) using optimal full matching that included applicant and application characteristics. Primary and subgroup analyses used generalized mixed models with obtaining any NIH R01 funding within 3 and 5 years as the two outcomes. A gamma sensitivity analysis was performed. URM applicants represented 11% and 12% of discussed and not discussed applications, respectively. First-time R01 applicants resubmitting their original, unfunded R01 application were more successful obtaining R01 funding within 3 and 5 years than applicants submitting new applications-for both discussed and not discussed applications: discussed within 3 years (OR 4.17 [95 CI 3.53, 4.93]) and 5 years (3.33 [2.82-3.92]); and not discussed within 3 years (2.81 [2.52, 3.13]) and 5 years (2.47 [2.22-2.74]). URM applicants additionally benefited within 5 years for not discussed applications. CONCLUSIONS: Encouraging early career researchers applying as faculty at a school of medicine to resubmit R01 applications is a promising potential modifiable factor and intervention strategy. First-time R01 applicants who resubmitted their original, unfunded R01 application had log-odds of obtaining downstream R01 funding within 3 and 5 years 2-4 times higher than applicants who did not resubmit their original application and submitted new NIH applications instead. Findings held for both discussed and not discussed applications.

Multiple United States Medical Licensing Examination Attempts and the Estimated Risk of Disciplinary Actions Among Graduates of U.S. and Canadian Medical Schools.

PURPOSE: The United States Medical Licensing Examination (USMLE) recently announced 2 policy changes: shifting from numeric score reporting on the Step 1 examination to pass/fail reporting and limiting examinees to 4 attempts for each Step component. In light of these policies, exam measures other than scores, such as the number of examination attempts, are of interest. Attempt limit policies are intended to ensure minimum standards of physician competency, yet little research has explored how Step attempts relate to physician practice outcomes. This study examined the relationship between USMLE attempts and the likelihood of receiving disciplinary actions from state medical boards. METHOD: The sample population was 219,018 graduates from U.S. and Canadian MD-granting medical schools who passed all USMLE Step examinations by 2011 and obtained a medical license in the United States, using data from the NBME and the Federation of State Medical Boards. Logistic regressions estimated how attempts on Steps 1, 2 Clinical Knowledge (CK), and 3 examinations influenced the likelihood of receiving disciplinary actions by 2018, while accounting for physician characteristics. RESULTS: A total of 3,399 physicians (2%) received at least 1 disciplinary action. Additional attempts needed to pass Steps 1, 2 CK, and 3 were associated with an increased likelihood of receiving disciplinary actions (odds ratio [OR]: 1.07, 95% confidence interval [CI]: 1.01, 1.13; OR: 1.09, 95% CI: 1.03, 1.16; OR: 1.11, 95% CI: 1.04, 1.17, respectively), after accounting for other factors. CONCLUSIONS: Physicians who took multiple attempts to pass Steps 1, 2 CK, and 3 were associated with higher estimated likelihood of receiving disciplinary actions. This study offers support for licensure and practice standards to account for physicians' USMLE attempts. The relatively small effect sizes, however, caution policy makers from placing sole emphasis on this relationship.

Farewell to the Step 2 Clinical Skills Exam: New Opportunities, Obligations, and Next Steps.

The unexpected discontinuation of the United States Medical Licensing Examination Step 2 Clinical Skills (CS) exam in January 2021 carries both risks and opportunities for medical education in the United States. Step 2 CS had far-reaching effects on medical school curricula and school-based clinical skills assessments. Absent the need to prepare students for this high-stakes exam, will the rigor of foundational clinical skills instruction and assessment remain a priority at medical schools? In this article, the authors consider the potential losses and gains from the elimination of Step 2 CS and explore opportunities to expand local summative assessments beyond the narrow bounds of Step 2 CS. The responsibility for implementing a rigorous and credible summative assessment of clinical skills that are critical for patient safety as medical students transition to residency now lies squarely with medical schools. Robust human simulation (standardized patient) programs, including regional and virtual simulation consortia, can provide infrastructure and expertise for innovative and creative local assessments to meet this need. Novel applications of human simulation and traditional formative assessment methods, such as workplace-based assessments and virtual patients, can contribute to defensible summative decisions about medical students' clinical skills. The need to establish validity evidence for decisions based on these novel assessment methods comprises a timely and relevant focus for medical education research.

Academic Medicine Faculty Perceptions of Work-Life Balance Before and Since the COVID-19 Pandemic.

Importance: How the COVID-19 pandemic has affected academic medicine faculty's work-life balance is unknown. Objective: To assess the association of perceived work-life conflict with academic medicine faculty intention to leave, reducing employment to part time, or declining leadership opportunities before and since the COVID-19 pandemic. Design, Settings, and Participants: An anonymous online survey of medical, graduate, and health professions school faculty was conducted at a single large, urban academic medical center between September 1 and September 25, 2020. Main Outcomes and Measures: Self-assessed intention to leave, reducing employment to part time, or turning down leadership opportunities because of work-life conflict before and since the COVID-19 pandemic. Results: Of the 1186 of 3088 (38%) of faculty members who answered the survey, 649 (55%) were women and 682 (58%) were White individuals. Respondents were representative of the overall faculty demographic characteristics except for an overrepresentation of female faculty respondents and underrepresentation of Asian faculty respondents compared with all faculty (female faculty: 649 [55%] vs 1368 [44%]; Asian faculty: 259 [22%] vs 963 [31%]). After the start of the COVID-19 pandemic, faculty were more likely to consider leaving or reducing employment to part time compared with before the pandemic (leaving: 225 [23%] vs 133 [14%]; P < .001; reduce hours: 281 [29%] vs 206 [22%]; P < .001). Women were more likely than men to reduce employment to part time before the COVID-19 pandemic (153 [28%] vs 44 [12%]; P < .001) and to consider both leaving or reducing employment to part time since the COVID-19 pandemic (leaving: 154 [28%] vs 56 [15%]; P < .001; reduce employment: 215 [40%] vs 49 [13%]; P < .001). Faculty with children were more likely to consider leaving and reducing employment since the COVID-19 pandemic compared with before the pandemic (leaving: 159 [29%] vs 93 [17%]; P < .001; reduce employment: 213 [40%] vs 130 [24%]; P < .001). Women with children compared with women without children were also more likely to consider leaving since the COVID-19 pandemic than before (113 [35%] vs 39 [17%]; P < .001). Working parent faculty and women were more likely to decline leadership opportunities both before (faculty with children vs without children: 297 [32%] vs 84 [9%]; P < .001; women vs men: 206 [29%] vs 47 [13%]; P < .001) and since the COVID-19 pandemic (faculty with children vs faculty without children: 316 [34%] vs 93 [10 %]; P < .001; women vs men: 148 [28%] vs 51 [14%]; P < .001). Conclusions and Relevance: In this survey study, the perceived stressors associated with work-life integration were higher in women than men, were highest in women with children, and have been exacerbated by the COVID-19 pandemic. The association of both gender and parenting with increased perceived work-life stress may disproportionately decrease the long-term retention and promotion of junior and midcareer women faculty.

The Dissolution of the Step 2 Clinical Skills Examination and the Duty of Medical Educators to Step Up the Effectiveness of Clinical Skills Assessment.

In this Invited Commentary, the authors explore the implications of the dissolution of the Step 2 Clinical Skills Examination (Step 2 CS) for medical student clinical skills assessment. The authors describe the need for medical educators (at both the undergraduate and graduate levels) to work collaboratively to improve medical student clinical skills assessment to assure the public that medical school graduates have the requisite skills to begin residency training. The authors outline 6 specific recommendations for how to capitalize on the discontinuation of Step 2 CS to improve clinical skills assessment: (1) defining national, end-of-clerkship, and transition-to-residency standards for required clinical skills and for levels of competence; (2) creating a national resource for standardized patient, augmented reality, and virtual reality assessments; (3) improving workplace-based assessment through local collaborations and national resources; (4) improving learner engagement in and coproduction of assessments; (5) requiring, as a new standard for accreditation, medical schools to establish and maintain competency committees; and (6) establishing a national registry of assessment data for research and evaluation. Together, these actions will help the medical education community earn the public's trust by enhancing the rigor of assessment to ensure the mastery of skills that are essential to providing safe, high-quality care for patients.

Using Natural Language Processing to Automatically Assess Feedback Quality: Findings From 3 Surgical Residencies.

PURPOSE: Learning is markedly improved with high-quality feedback, yet assuring the quality of feedback is difficult to achieve at scale. Natural language processing (NLP) algorithms may be useful in this context as they can automatically classify large volumes of narrative data. However, it is unknown if NLP models can accurately evaluate surgical trainee feedback. This study evaluated which NLP techniques best classify the quality of surgical trainee formative feedback recorded as part of a workplace assessment. METHOD: During the 2016-2017 academic year, the SIMPL (Society for Improving Medical Professional Learning) app was used to record operative performance narrative feedback for residents at 3 university-based general surgery residency training programs. Feedback comments were collected for a sample of residents representing all 5 postgraduate year levels and coded for quality. In May 2019, the coded comments were then used to train NLP models to automatically classify the quality of feedback across 4 categories (effective, mediocre, ineffective, or other). Models included support vector machines (SVM), logistic regression, gradient boosted trees, naive Bayes, and random forests. The primary outcome was mean classification accuracy. RESULTS: The authors manually coded the quality of 600 recorded feedback comments. Those data were used to train NLP models to automatically classify the quality of feedback across 4 categories. The NLP model using an SVM algorithm yielded a maximum mean accuracy of 0.64 (standard deviation, 0.01). When the classification task was modified to distinguish only high-quality vs low-quality feedback, maximum mean accuracy was 0.83, again with SVM. CONCLUSIONS: To the authors' knowledge, this is the first study to examine the use of NLP for classifying feedback quality. SVM NLP models demonstrated the ability to automatically classify the quality of surgical trainee evaluations. Larger training datasets would likely further increase accuracy.

The attitudes, perceptions and experiences of medical school applicants following the closure of schools and cancellation of public examinations in 2020 due to the COVID-19 pandemic: a cross-sectional questionnaire study of UK medical applicants.

OBJECTIVE: Describe the experiences and views of medical applicants from diverse social backgrounds following the closure of schools and universities and the cancellation of public examinations in the UK due to COVID-19. DESIGN: Cross-sectional questionnaire study, part of the longitudinal UK Medical Applicant Cohort Study (UKMACS). SETTING: UK medical school admissions in 2020. PARTICIPANTS: 2887 participants completed an online questionnaire from 8 April to 22 April 2020. Eligible participants had registered to take the University Clinical Admissions Test in 2019 and agreed to be invited to take part, or had completed a previous UKMACS questionnaire, had been seriously considering applying to medicine in the UK for entry in 2020, and were UK residents. MAIN OUTCOME MEASURES: Views on calculated grades, views on medical school admissions and teaching in 2020 and 2021, reported experiences of education during the national lockdown. RESULTS: Respondents were concerned about the calculated grades that replaced A-level examinations: female and Black Asian and Minority Ethnic applicants felt teachers would find it difficult to grade and rank students accurately, and applicants from non-selective state schools and living in deprived areas had concerns about the standardisation process. Calculated grades were generally not considered fair enough to use in selection, but were considered fair enough to use in combination with other measures including interview and aptitude test scores. Respondents from non-selective state (public) schools reported less access to educational resources compared with private/selective school pupils, less online teaching in real time and less time studying during lockdown. CONCLUSIONS: The COVID-19 pandemic has and will have significant and long-term impacts on the selection, education and performance of our medical workforce. It is important that the views and experiences of applicants from diverse backgrounds are considered in decisions affecting their future and the future of the profession.

Medical School Accreditation Factors Associated With Certification by the Educational Commission for Foreign Medical Graduates (ECFMG): A 10-Year International Study.

PURPOSE: In 2024, international medical graduates seeking Educational Commission for Foreign Medical Graduates (ECFMG) certification will be required to graduate from an accredited medical school. This study's goal was to examine relationships between medical school accreditation variables and ECFMG certification for a global sample. METHOD: Using ECFMG databases, the authors created a 10-year cohort (January 1, 2007-December 31, 2016) of certification applicants, defined as individuals who had attempted at least 2 examinations required for certification. The authors aggregated applicant data at the school level, excluding schools with < 80 applicants. School accreditation statuses were based on agency websites. School region, age, and time of first accreditation were included. Analyses included descriptive and bivariate statistics and multiple linear regressions adjusting for school start year and year of first accreditation. RESULTS: The cohort included 128,046 applicants from 1,973 medical schools across 162 countries. After excluding low-volume schools, 318 schools across 81 countries remained. These provided 99,598 applicants and 77,919 certificate holders, three-quarters of whom came from the Caribbean, South-Central Asia, and West Asia regions. Two hundred and fifty (78.6%) schools were accredited; 68 (21.4%) were not. Most ECFMG applicants (n = 84,776, 85.1%) and certificate holders (n = 68,444, 87.8%) attended accredited medical schools. Accredited schools had higher rates of ECFMG certification among graduates than nonaccredited schools in comparisons that included all schools (75.0% [standard deviation (SD) = 10.6%] vs 68.3% (SD = 15.9%), P < .001), and for countries that had both accredited and nonaccredited schools (73.9% [SD = 11.4%] vs 67.3% [SD = 16.8%], P = .023). After adjusting for age of school, longer duration of accreditation was associated with higher certification rates (P < .001). CONCLUSIONS: Accreditation had a positive association with ECFMG certification rates. Future studies should investigate how accreditation processes might account for higher certification rates.

EurOP2E - the European Open Platform for Prescribing Education, a consensus study among clinical pharmacology and therapeutics teachers.

PURPOSE: Sharing and developing digital educational resources and open educational resources has been proposed as a way to harmonize and improve clinical pharmacology and therapeutics (CPT) education in European medical schools. Previous research, however, has shown that there are barriers to the adoption and implementation of open educational resources. The aim of this study was to determine perceived opportunities and barriers to the use and creation of open educational resources among European CPT teachers and possible solutions for these barriers. METHODS: CPT teachers of British and EU medical schools completed an online survey. Opportunities and challenges were identified by thematic analyses and subsequently discussed in an international consensus meeting. RESULTS: Data from 99 CPT teachers from 95 medical schools were analysed. Thirty teachers (30.3%) shared or collaboratively produced digital educational resources. All teachers foresaw opportunities in the more active use of open educational resources, including improving the quality of their teaching. The challenges reported were language barriers, local differences, lack of time, technological issues, difficulties with quality management, and copyright restrictions. Practical solutions for these challenges were discussed and include a peer review system, clear indexing, and use of copyright licenses that permit adaptation of resources. CONCLUSION: Key challenges to making greater use of CPT open educational resources are a limited applicability of such resources due to language and local differences and quality concerns. These challenges may be resolved by relatively simple measures, such as allowing adaptation and translation of resources and a peer review system.

Mental Health from Medical School to Medical Practice: Finding a Path Forward.

Medical students, residents, and practicing physicians experience high burnout, depression, and suicide rates, and the COVID-19 pandemic has exacerbated stress for many.1-6 While laudable, current well-being efforts appear insufficient to meet the challenges that so many are facing. This essay explores approaches that individuals and organizations can take to promote mental health and well-being from medical school to practice.

Mission matters: Association between a medical school's mission and minority student representation.

Increasing enrollment of students who are underrepresented in medicine has been a priority of United States (US) medical schools. The authors sought to compare how increasing minority student representation factors into mission statements, statements of values, and strategic action plans at top research-oriented US medical schools and US medical schools with a social mission. A Web search was performed to locate three documents for each medical school: the mission statement; a statement of values; and a strategic plan. Data were retrieved on the number of underrepresented minority graduates and total graduates from each school in the graduating classes of 2015-2019. The number and percentage of graduates during this period were compared according to schools' mission statements using rank-sum tests. Other quantitative study data were compared by school mission using Fisher's exact tests. Five of the schools with a social mission (25%) and none of the schools with a research mission had a mission statement that addressed increasing representation of underrepresented minority students in the medical school (p = 0.047). Schools with a mission statement that addressed this group had a higher proportion of those graduates during 2015-2019 (median 66%; IQR 28%, 68%) compared to schools that did not address this in their mission statement (median 10%; IQR 6%, 13%; p = 0.003). More research is needed to explore the association between US medical school mission statements and the representation of underrepresented students in medical education, especially at research-oriented medical schools.